Pandas Profiling Report

Dataset statistics

Number of variables	13
Number of observations	29101
Missing cells	3043
Missing cells (%)	0.8%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	2.9 MiB
Average record size in memory	104.0 B

Variable types

Categorical	2
Numeric	10
Boolean	1

Warnings

`pickup_dt` has a high cardinality: 4343 distinct values	High cardinality
`temp` is highly correlated with `dewp` and 1 other fields	High correlation
`dewp` is highly correlated with `temp`	High correlation
`sd` is highly correlated with `temp`	High correlation
`vsb` is highly correlated with `pcp01`	High correlation
`temp` is highly correlated with `dewp` and 1 other fields	High correlation
`dewp` is highly correlated with `temp` and 1 other fields	High correlation
`pcp01` is highly correlated with `vsb`	High correlation
`sd` is highly correlated with `temp` and 1 other fields	High correlation
`vsb` is highly correlated with `pcp01`	High correlation
`temp` is highly correlated with `dewp`	High correlation
`dewp` is highly correlated with `temp`	High correlation
`pcp01` is highly correlated with `vsb`	High correlation
`pcp01` is highly correlated with `vsb`	High correlation
`sd` is highly correlated with `dewp` and 2 other fields	High correlation
`vsb` is highly correlated with `pcp01`	High correlation
`borough` is highly correlated with `pickups`	High correlation
`dewp` is highly correlated with `sd` and 2 other fields	High correlation
`pickups` is highly correlated with `borough`	High correlation
`temp` is highly correlated with `sd` and 1 other fields	High correlation
`slp` is highly correlated with `sd` and 1 other fields	High correlation
`borough` has 3043 (10.5%) missing values	Missing
`pickup_dt` is uniformly distributed	Uniform
`borough` is uniformly distributed	Uniform
`pickups` has 5567 (19.1%) zeros	Zeros
`spd` has 3596 (12.4%) zeros	Zeros
`dewp` has 303 (1.0%) zeros	Zeros
`pcp01` has 26468 (91.0%) zeros	Zeros
`pcp06` has 23460 (80.6%) zeros	Zeros
`pcp24` has 18631 (64.0%) zeros	Zeros
`sd` has 20167 (69.3%) zeros	Zeros

Reproduction

Analysis started	2021-08-14 17:30:19.448068
Analysis finished	2021-08-14 17:30:42.636455
Duration	23.19 seconds
Software version	pandas-profiling v3.0.0
Download configuration	config.json

pickup_dt
Categorical

HIGH CARDINALITY
UNIFORM

Distinct	4343
Distinct (%)	14.9%
Missing	0
Missing (%)	0.0%
Memory size	227.5 KiB

2015-02-15 16:00:00	7
2015-06-11 04:00:00	7
2015-04-20 21:00:00	7
2015-01-23 01:00:00	7
2015-05-31 16:00:00	7
Other values (4338)	29066

Length

Max length	19
Median length	19
Mean length	19
Min length	19

Characters and Unicode

Total characters	552919
Distinct characters	13
Distinct categories	4 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	2015-01-01 01:00:00
2nd row	2015-01-01 01:00:00
3rd row	2015-01-01 01:00:00
4th row	2015-01-01 01:00:00
5th row	2015-01-01 01:00:00

Common Values

Value	Count	Frequency (%)
2015-02-15 16:00:00	7	< 0.1%
2015-06-11 04:00:00	7	< 0.1%
2015-04-20 21:00:00	7	< 0.1%
2015-01-23 01:00:00	7	< 0.1%
2015-05-31 16:00:00	7	< 0.1%
2015-03-17 06:00:00	7	< 0.1%
2015-06-22 11:00:00	7	< 0.1%
2015-05-06 09:00:00	7	< 0.1%
2015-02-27 14:00:00	7	< 0.1%
2015-06-12 21:00:00	7	< 0.1%
Other values (4333)	29031	99.8%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
22:00:00	1237	2.1%
09:00:00	1236	2.1%
19:00:00	1233	2.1%
20:00:00	1233	2.1%
07:00:00	1232	2.1%
21:00:00	1230	2.1%
08:00:00	1229	2.1%
17:00:00	1228	2.1%
16:00:00	1226	2.1%
10:00:00	1225	2.1%
Other values (195)	45893	78.9%

Most occurring characters

Value	Count	Frequency (%)
0	201596	36.5%
1	62876	11.4%
-	58202	10.5%
:	58202	10.5%
2	54544	9.9%
5	39446	7.1%
	29101	5.3%
3	12735	2.3%
6	10236	1.9%
4	10059	1.8%
Other values (3)	15922	2.9%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	407414	73.7%
Dash Punctuation	58202	10.5%
Other Punctuation	58202	10.5%
Space Separator	29101	5.3%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
0	201596	49.5%
1	62876	15.4%
2	54544	13.4%
5	39446	9.7%
3	12735	3.1%
6	10236	2.5%
4	10059	2.5%
7	5358	1.3%
8	5357	1.3%
9	5207	1.3%

Dash Punctuation

Value	Count	Frequency (%)
-	58202	100.0%

Space Separator

Value	Count	Frequency (%)
	29101	100.0%

Other Punctuation

Value	Count	Frequency (%)
:	58202	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	552919	100.0%

Most frequent character per script

Common

Value	Count	Frequency (%)
0	201596	36.5%
1	62876	11.4%
-	58202	10.5%
:	58202	10.5%
2	54544	9.9%
5	39446	7.1%
	29101	5.3%
3	12735	2.3%
6	10236	1.9%
4	10059	1.8%
Other values (3)	15922	2.9%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	552919	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
0	201596	36.5%
1	62876	11.4%
-	58202	10.5%
:	58202	10.5%
2	54544	9.9%
5	39446	7.1%
	29101	5.3%
3	12735	2.3%
6	10236	1.9%
4	10059	1.8%
Other values (3)	15922	2.9%

borough
Categorical

HIGH CORRELATION
MISSING
UNIFORM

Distinct	6
Distinct (%)	< 0.1%
Missing	3043
Missing (%)	10.5%
Memory size	227.5 KiB

EWR	4343
Staten Island	4343
Bronx	4343
Brooklyn	4343
Manhattan	4343

Length

Max length	13
Median length	7
Mean length	7.333333333
Min length	3

Characters and Unicode

Total characters	191092
Distinct characters	23
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	Bronx
2nd row	Brooklyn
3rd row	EWR
4th row	Manhattan
5th row	Queens

Common Values

Value	Count	Frequency (%)
EWR	4343	14.9%
Staten Island	4343	14.9%
Bronx	4343	14.9%
Brooklyn	4343	14.9%
Manhattan	4343	14.9%
Queens	4343	14.9%
(Missing)	3043	10.5%

Length

Histogram of lengths of the category

Pie chart

Value	Count	Frequency (%)
island	4343	14.3%
queens	4343	14.3%
brooklyn	4343	14.3%
staten	4343	14.3%
ewr	4343	14.3%
bronx	4343	14.3%
manhattan	4343	14.3%

Most occurring characters

Value	Count	Frequency (%)
n	30401	15.9%
a	21715	11.4%
t	17372	9.1%
o	13029	6.8%
e	13029	6.8%
B	8686	4.5%
r	8686	4.5%
l	8686	4.5%
s	8686	4.5%
x	4343	2.3%
Other values (13)	56459	29.5%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	147662	77.3%
Uppercase Letter	39087	20.5%
Space Separator	4343	2.3%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
n	30401	20.6%
a	21715	14.7%
t	17372	11.8%
o	13029	8.8%
e	13029	8.8%
r	8686	5.9%
l	8686	5.9%
s	8686	5.9%
x	4343	2.9%
k	4343	2.9%
Other values (4)	17372	11.8%

Uppercase Letter

Value	Count	Frequency (%)
B	8686	22.2%
E	4343	11.1%
W	4343	11.1%
R	4343	11.1%
M	4343	11.1%
Q	4343	11.1%
S	4343	11.1%
I	4343	11.1%

Space Separator

Value	Count	Frequency (%)
	4343	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	186749	97.7%
Common	4343	2.3%

Most frequent character per script

Latin

Value	Count	Frequency (%)
n	30401	16.3%
a	21715	11.6%
t	17372	9.3%
o	13029	7.0%
e	13029	7.0%
B	8686	4.7%
r	8686	4.7%
l	8686	4.7%
s	8686	4.7%
x	4343	2.3%
Other values (12)	52116	27.9%

Common

Value	Count	Frequency (%)
	4343	100.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	191092	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
n	30401	15.9%
a	21715	11.4%
t	17372	9.1%
o	13029	6.8%
e	13029	6.8%
B	8686	4.5%
r	8686	4.5%
l	8686	4.5%
s	8686	4.5%
x	4343	2.3%
Other values (13)	56459	29.5%

pickups
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	3406
Distinct (%)	11.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	490.2159032

Minimum	0
Maximum	7883
Zeros	5567
Zeros (%)	19.1%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	1
median	54
Q3	449
95-th percentile	2840
Maximum	7883
Range	7883
Interquartile range (IQR)	448

Descriptive statistics

Standard deviation	995.6495355
Coefficient of variation (CV)	2.031042912
Kurtosis	9.26766556
Mean	490.2159032
Median Absolute Deviation (MAD)	54
Skewness	2.976238116
Sum	14265773
Variance	991317.9975
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	5567	19.1%
1	2656	9.1%
2	1698	5.8%
3	937	3.2%
4	474	1.6%
5	257	0.9%
6	128	0.4%
36	85	0.3%
45	84	0.3%
32	81	0.3%
Other values (3396)	17134	58.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	5567	19.1%
1	2656	9.1%
2	1698	5.8%
3	937	3.2%
4	474	1.6%
5	257	0.9%
6	128	0.4%
7	77	0.3%
8	45	0.2%
9	46	0.2%

Value	Count	Frequency (%)
7883	1	< 0.1%
7801	1	< 0.1%
7711	1	< 0.1%
7512	1	< 0.1%
7271	1	< 0.1%
7240	1	< 0.1%
7140	1	< 0.1%
7114	1	< 0.1%
7046	1	< 0.1%
6976	1	< 0.1%

spd
Real number (ℝ_≥0)

ZEROS

Distinct	114
Distinct (%)	0.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	5.98492418

Minimum	0
Maximum	21
Zeros	3596
Zeros (%)	12.4%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	3
median	6
Q3	8
95-th percentile	13
Maximum	21
Range	21
Interquartile range (IQR)	5

Descriptive statistics

Standard deviation	3.699007242
Coefficient of variation (CV)	0.6180541525
Kurtosis	0.4192409725
Mean	5.98492418
Median Absolute Deviation (MAD)	2
Skewness	0.4190693213
Sum	174167.2786
Variance	13.68265458
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
5	3816	13.1%
0	3596	12.4%
6	3545	12.2%
3	3432	11.8%
7	3021	10.4%
8	2574	8.8%
9	1592	5.5%
10	1369	4.7%
11	936	3.2%
13	557	1.9%
Other values (104)	4663	16.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	3596	12.4%
0.6	21	0.1%
0.75	42	0.1%
1	66	0.2%
1.2	21	0.1%
1.285714286	7	< 0.1%
1.5	256	0.9%
1.6	7	< 0.1%
1.666666667	35	0.1%
1.8	14	< 0.1%

Value	Count	Frequency (%)
21	7	< 0.1%
20	32	0.1%
18	71	0.2%
17.5	7	< 0.1%
17	120	0.4%
16.75	6	< 0.1%
16.5	7	< 0.1%
16.33333333	6	< 0.1%
16	183	0.6%
15.75	12	< 0.1%

vsb
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	179
Distinct (%)	0.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	8.818124897

Minimum	0
Maximum	10
Zeros	6
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	2.575
Q1	9.1
median	10
Q3	10
95-th percentile	10
Maximum	10
Range	10
Interquartile range (IQR)	0.9

Descriptive statistics

Standard deviation	2.442897359
Coefficient of variation (CV)	0.2770313856
Kurtosis	2.898539633
Mean	8.818124897
Median Absolute Deviation (MAD)	0
Skewness	-2.042058313
Sum	256616.2526
Variance	5.967747505
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10	21578	74.1%
9.1	1137	3.9%
8	845	2.9%
7	780	2.7%
6	560	1.9%
4	403	1.4%
5	395	1.4%
3	267	0.9%
0.3	127	0.4%
3.5	108	0.4%
Other values (169)	2901	10.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	6	< 0.1%
0.3	127	0.4%
0.3333333333	6	< 0.1%
0.3666666667	7	< 0.1%
0.4	20	0.1%
0.4333333333	7	< 0.1%
0.5	60	0.2%
0.6	20	0.1%
0.65	14	< 0.1%
0.6666666667	7	< 0.1%

Value	Count	Frequency (%)
10	21578	74.1%
9.775	7	< 0.1%
9.7	20	0.1%
9.55	72	0.2%
9.333333333	14	< 0.1%
9.1	1137	3.9%
8.733333333	12	< 0.1%
8.55	33	0.1%
8.5	7	< 0.1%
8.4	6	< 0.1%

temp
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	295
Distinct (%)	1.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	47.66904206

Minimum	2
Maximum	89
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	2
5-th percentile	18
Q1	32
median	46
Q3	64.5
95-th percentile	79.5
Maximum	89
Range	87
Interquartile range (IQR)	32.5

Descriptive statistics

Standard deviation	19.81496901
Coefficient of variation (CV)	0.4156779359
Kurtosis	-1.037412126
Mean	47.66904206
Median Absolute Deviation (MAD)	16
Skewness	0.05575251227
Sum	1387216.793
Variance	392.6329968
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
37	675	2.3%
36	595	2.0%
35	548	1.9%
42	531	1.8%
38	529	1.8%
34	514	1.8%
27	508	1.7%
61	502	1.7%
39	494	1.7%
41	494	1.7%
Other values (285)	23711	81.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2	20	0.1%
3	14	< 0.1%
4	85	0.3%
5	33	0.1%
6	41	0.1%
7	28	0.1%
8	53	0.2%
9	65	0.2%
10	53	0.2%
11	44	0.2%

Value	Count	Frequency (%)
89	28	0.1%
88	56	0.2%
87	54	0.2%
86	61	0.2%
85	144	0.5%
84	212	0.7%
83	159	0.5%
82	289	1.0%
81.5	7	< 0.1%
81	199	0.7%

dewp
Real number (ℝ)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
ZEROS

Distinct	305
Distinct (%)	1.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	30.82306491

Minimum	-16
Maximum	73
Zeros	303
Zeros (%)	1.0%
Negative	1950
Negative (%)	6.7%
Memory size	227.5 KiB

Quantile statistics

Minimum	-16
5-th percentile	-2
Q1	14
median	30
Q3	50
95-th percentile	64.66666667
Maximum	73
Range	89
Interquartile range (IQR)	36

Descriptive statistics

Standard deviation	21.28344434
Coefficient of variation (CV)	0.6905038288
Kurtosis	-1.035223571
Mean	30.82306491
Median Absolute Deviation (MAD)	18
Skewness	0.0154181971
Sum	896982.0119
Variance	452.9850028
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
39	578	2.0%
22	525	1.8%
18	518	1.8%
56	508	1.7%
61	492	1.7%
25	486	1.7%
40	473	1.6%
20	465	1.6%
10	460	1.6%
60	450	1.5%
Other values (295)	24146	83.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-16	34	0.1%
-15	27	0.1%
-13	86	0.3%
-12	98	0.3%
-11	121	0.4%
-10	53	0.2%
-9	100	0.3%
-8	92	0.3%
-7	112	0.4%
-6	155	0.5%

Value	Count	Frequency (%)
73	7	< 0.1%
72	14	< 0.1%
71.5	12	< 0.1%
71.25	7	< 0.1%
71	70	0.2%
70.33333333	7	< 0.1%
70.28571429	6	< 0.1%
70	62	0.2%
69.66666667	6	< 0.1%
69.33333333	6	< 0.1%

slp
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	413
Distinct (%)	1.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	1017.817938

Minimum	991.4
Maximum	1043.4
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	991.4
5-th percentile	1005.3
Q1	1012.5
median	1018.2
Q3	1022.9
95-th percentile	1030
Maximum	1043.4
Range	52
Interquartile range (IQR)	10.4

Descriptive statistics

Standard deviation	7.76879558
Coefficient of variation (CV)	0.007632794917
Kurtosis	0.06914463865
Mean	1017.817938
Median Absolute Deviation (MAD)	5.2
Skewness	0.05284461782
Sum	29619519.8
Variance	60.35418476
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1020	269	0.9%
1020.5	255	0.9%
1019.9	237	0.8%
1020.9	227	0.8%
1021.1	226	0.8%
1022.7	214	0.7%
1020.2	213	0.7%
1020.3	209	0.7%
1021.2	204	0.7%
1020.7	204	0.7%
Other values (403)	26843	92.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
991.4	7	< 0.1%
991.6	7	< 0.1%
992.3	7	< 0.1%
992.9	7	< 0.1%
993.4	7	< 0.1%
993.7	7	< 0.1%
994.1	7	< 0.1%
995.3	20	0.1%
996.2	7	< 0.1%
996.4	6	< 0.1%

Value	Count	Frequency (%)
1043.4	7	< 0.1%
1043.3	7	< 0.1%
1043.2	7	< 0.1%
1043.1	7	< 0.1%
1042.9	6	< 0.1%
1042.2	7	< 0.1%
1041.7	7	< 0.1%
1041.6	7	< 0.1%
1041.2	7	< 0.1%
1041.1	14	< 0.1%

pcp01
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
ZEROS

Distinct	80
Distinct (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	0.003830149021

Minimum	0
Maximum	0.28
Zeros	26468
Zeros (%)	91.0%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0
95-th percentile	0.02
Maximum	0.28
Range	0.28
Interquartile range (IQR)	0

Descriptive statistics

Standard deviation	0.01893306515
Coefficient of variation (CV)	4.943166713
Kurtosis	87.81998828
Mean	0.003830149021
Median Absolute Deviation (MAD)	0
Skewness	8.220954559
Sum	111.4611667
Variance	0.0003584609559
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	26468	91.0%
0.01	439	1.5%
0.02	181	0.6%
0.03	147	0.5%
0.005	141	0.5%
0.05	115	0.4%
0.003333333333	95	0.3%
0.04	79	0.3%
0.015	78	0.3%
0.06	75	0.3%
Other values (70)	1283	4.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	26468	91.0%
0.0025	40	0.1%
0.003333333333	95	0.3%
0.005	141	0.5%
0.006666666667	62	0.2%
0.0075	33	0.1%
0.008	13	< 0.1%
0.01	439	1.5%
0.01166666667	7	< 0.1%
0.012	6	< 0.1%

Value	Count	Frequency (%)
0.28	21	0.1%
0.2675	7	< 0.1%
0.26	7	< 0.1%
0.2533333333	7	< 0.1%
0.25	7	< 0.1%
0.2475	7	< 0.1%
0.198	6	< 0.1%
0.19	6	< 0.1%
0.17	13	< 0.1%
0.15	27	0.1%

pcp06
Real number (ℝ_≥0)

ZEROS

Distinct	318
Distinct (%)	1.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	0.02612874128

Minimum	0
Maximum	1.24
Zeros	23460
Zeros (%)	80.6%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0
95-th percentile	0.1875
Maximum	1.24
Range	1.24
Interquartile range (IQR)	0

Descriptive statistics

Standard deviation	0.09312533965
Coefficient of variation (CV)	3.564095899
Kurtosis	47.35606139
Mean	0.02612874128
Median Absolute Deviation (MAD)	0
Skewness	5.936438429
Sum	760.3725
Variance	0.008672328884
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	23460	80.6%
0.01	841	2.9%
0.02	258	0.9%
0.03	177	0.6%
0.05	152	0.5%
0.005	121	0.4%
0.04	110	0.4%
0.06	95	0.3%
0.08	92	0.3%
0.003333333333	87	0.3%
Other values (308)	3708	12.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	23460	80.6%
0.0025	60	0.2%
0.003333333333	87	0.3%
0.005	121	0.4%
0.006666666667	33	0.1%
0.0075	28	0.1%
0.008	7	< 0.1%
0.008333333333	7	< 0.1%
0.01	841	2.9%
0.01333333333	13	< 0.1%

Value	Count	Frequency (%)
1.24	6	< 0.1%
1.22	6	< 0.1%
1.21	7	< 0.1%
1.083	7	< 0.1%
1.018	7	< 0.1%
1.0105	7	< 0.1%
0.895	7	< 0.1%
0.875	7	< 0.1%
0.86	7	< 0.1%
0.835	7	< 0.1%

pcp24
Real number (ℝ_≥0)

ZEROS

Distinct	484
Distinct (%)	1.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	0.09046437121

Minimum	0
Maximum	2.1
Zeros	18631
Zeros (%)	64.0%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0.05
95-th percentile	0.5755
Maximum	2.1
Range	2.1
Interquartile range (IQR)	0.05

Descriptive statistics

Standard deviation	0.2194022017
Coefficient of variation (CV)	2.425288528
Kurtosis	16.22082135
Mean	0.09046437121
Median Absolute Deviation (MAD)	0
Skewness	3.605783873
Sum	2632.603667
Variance	0.04813732611
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	18631	64.0%
0.01	1259	4.3%
0.05	367	1.3%
0.02	306	1.1%
0.09	271	0.9%
0.08333333333	256	0.9%
0.06	225	0.8%
0.08	217	0.7%
0.1793333333	152	0.5%
0.03	149	0.5%
Other values (474)	7268	25.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	18631	64.0%
0.0025	72	0.2%
0.003333333333	91	0.3%
0.005	87	0.3%
0.005833333333	107	0.4%
0.006666666667	19	0.1%
0.0075	21	0.1%
0.008	7	< 0.1%
0.01	1259	4.3%
0.015	52	0.2%

Value	Count	Frequency (%)
2.1	13	< 0.1%
1.89	7	< 0.1%
1.503833333	64	0.2%
1.493833333	7	< 0.1%
1.49	7	< 0.1%
1.488833333	21	0.1%
1.46	6	< 0.1%
1.42	6	< 0.1%
1.418833333	7	< 0.1%
1.413833333	7	< 0.1%

sd
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
ZEROS

Distinct	421
Distinct (%)	1.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	2.529169244

Minimum	0
Maximum	19
Zeros	20167
Zeros (%)	69.3%
Negative	0
Negative (%)	0.0%
Memory size	227.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	2.958333333
95-th percentile	12.16666667
Maximum	19
Range	19
Interquartile range (IQR)	2.958333333

Descriptive statistics

Standard deviation	4.520325424
Coefficient of variation (CV)	1.787276765
Kurtosis	1.313944097
Mean	2.529169244
Median Absolute Deviation (MAD)	0
Skewness	1.589743978
Sum	73601.35417
Variance	20.43334194
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	20167	69.3%
8	1934	6.6%
11	362	1.2%
12	345	1.2%
9	334	1.1%
7	182	0.6%
1	181	0.6%
13	180	0.6%
2	175	0.6%
0.875	40	0.1%
Other values (411)	5201	17.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	20167	69.3%
0.04166666667	19	0.1%
0.04166666667	13	< 0.1%
0.08333333333	13	< 0.1%
0.08333333333	19	0.1%
0.125	39	0.1%
0.1666666667	32	0.1%
0.2083333333	33	0.1%
0.25	39	0.1%
0.2916666667	34	0.1%

Value	Count	Frequency (%)
19	7	< 0.1%
18.95833333	7	< 0.1%
18.91666667	6	< 0.1%
18.875	6	< 0.1%
18.83333333	7	< 0.1%
18.79166667	7	< 0.1%
18.75	14	< 0.1%
18.70833333	6	< 0.1%
18.66666667	7	< 0.1%
18.625	7	< 0.1%

hday
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	28.5 KiB

False	27980
True	1121

Common Values
Chart

Value	Count	Frequency (%)
False	27980	96.1%
True	1121	3.9%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	pickup_dt	borough	pickups	spd	vsb	temp	dewp	slp	hday
0	2015-01-01 01:00:00	Bronx	152	5.0	10.0	30.0	7.0	1023.5	Y
1	2015-01-01 01:00:00	Brooklyn	1519	5.0	10.0	30.0	7.0	1023.5	Y
2	2015-01-01 01:00:00	EWR	0	5.0	10.0	30.0	7.0	1023.5	Y
3	2015-01-01 01:00:00	Manhattan	5258	5.0	10.0	30.0	7.0	1023.5	Y
4	2015-01-01 01:00:00	Queens	405	5.0	10.0	30.0	7.0	1023.5	Y
5	2015-01-01 01:00:00	Staten Island	6	5.0	10.0	30.0	7.0	1023.5	Y
6	2015-01-01 01:00:00	NaN	4	5.0	10.0	30.0	7.0	1023.5	Y
7	2015-01-01 02:00:00	Bronx	120	3.0	10.0	30.0	6.0	1023.0	Y
8	2015-01-01 02:00:00	Brooklyn	1229	3.0	10.0	30.0	6.0	1023.0	Y
9	2015-01-01 02:00:00	EWR	0	3.0	10.0	30.0	6.0	1023.0	Y

Last rows

	pickup_dt	borough	pickups	spd	vsb	temp	dewp	slp	hday
29091	2015-06-30 22:00:00	Manhattan	4452	5.0	10.0	76.0	64.0	1011.9	N
29092	2015-06-30 22:00:00	Queens	556	5.0	10.0	76.0	64.0	1011.9	N
29093	2015-06-30 22:00:00	Staten Island	2	5.0	10.0	76.0	64.0	1011.9	N
29094	2015-06-30 23:00:00	Bronx	67	7.0	10.0	75.0	65.0	1011.8	N
29095	2015-06-30 23:00:00	Brooklyn	990	7.0	10.0	75.0	65.0	1011.8	N
29096	2015-06-30 23:00:00	EWR	0	7.0	10.0	75.0	65.0	1011.8	N
29097	2015-06-30 23:00:00	Manhattan	3828	7.0	10.0	75.0	65.0	1011.8	N
29098	2015-06-30 23:00:00	Queens	580	7.0	10.0	75.0	65.0	1011.8	N
29099	2015-06-30 23:00:00	Staten Island	0	7.0	10.0	75.0	65.0	1011.8	N
29100	2015-06-30 23:00:00	NaN	3	7.0	10.0	75.0	65.0	1011.8	N

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Decimal Number

Dash Punctuation

Space Separator

Other Punctuation

Most occurring scripts

Most frequent character per script

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Pie chart

Most occurring characters

Most occurring categories

Most frequent character per category

Lowercase Letter

Uppercase Letter

Space Separator

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows